Fixed validation bug where generator expression was not being evaluated.

Changed from checking generator object to using all() to properly validate
all messages are instances of HumanMessage, AIMessage, or ToolMessage.

Added tests to verify validation works correctly.

Problem Description

Problem: The MultiTurnSample.validate_user_input() method had a critical validation bug where the generator expression was not being properly evaluated. The code was checking if not (isinstance(m, ...) for m in messages): which creates a generator object that is always truthy, causing the validation to never trigger.

Impact: This meant that invalid message types could potentially pass validation if they somehow bypassed Pydantic's type checking, though in practice Pydantic's Union validation catches most cases before this validator runs. However, the validator logic itself was fundamentally broken and would not work correctly if called.

How to replicate: The bug can be seen in the code at src/ragas/dataset_schema.py:131-133 where the generator expression without all() would never properly validate the messages.

Changes Made

• Fixed validation logic in src/ragas/dataset_schema.py: Changed if not (isinstance(m, ...) for m in messages): to if not all(isinstance(m, ...) for m in messages): to properly evaluate all message type checks
• Added comprehensive tests in tests/unit/test_dataset_schema.py:
  • test_multiturn_sample_validate_user_input_invalid_type(): Verifies that invalid message types are properly rejected
  • test_multiturn_sample_validate_user_input_valid_types(): Verifies that valid message types are properly accepted

References

• File changed: src/ragas/dataset_schema.py (line 131-133)
• Tests added: tests/unit/test_dataset_schema.py (lines 201-226)
• Related code: The validator is part of the MultiTurnSample class which is used throughout the codebase for multi-turn conversation evaluation